meysam sadeghi; mohammadreza falsafinejad; ali delavar; noorali farrokhi; ehsan jamali
Abstract
The purpose of the present study was to identify the weight of each of the courses of academic record and scoring based on the psychometric approach of the IRT continuous model and approaches based on the viewpoint of the specialists of Topsis and AHP. The method of this study was a combination. In order ...
Read More
The purpose of the present study was to identify the weight of each of the courses of academic record and scoring based on the psychometric approach of the IRT continuous model and approaches based on the viewpoint of the specialists of Topsis and AHP. The method of this study was a combination. In order to conduct research in the first part, 11 items (courses) of academic background were taken from the expert group and weighed and prioritized using Tapis and AHP techniques. The final third year grades of secondary school students were received from the assessment organization and accepted for psychology and counseling in the humanities group. Theta tests were used to estimate the grades. In order to compare different weighting and grading models, the sample group was first ranked based on scores derived from all three approaches, and compared in terms of dispersion indices and the degree of difference between rankings. The results of the comparison of the three approaches indicated that the most variance is related to the IRT continuous modeling method. The results of Wilcoxon's statistical analysis to compare the average rankings showed that in the small sample size, the weighting method in all three methods produced a significant difference in the rankings compared to the same weight method, and therefore the weighting model in the ranking of the volunteers In small sample size affects. But there are no significant differences between the three weighting models in the candidate ranking. Also, the results showed that in the high sample size (2000 persons), the weighting method in all three methods compared to the same weights makes a significant difference in the rankings. IRT method scores are more reliable than expert-based approaches.However, the weighting method does not seem to have a significant effect on the reliability of the scores.
negar sharifi; mohammad falsafi; noorali farokhi; ehsan jamali
Abstract
Background: Test fairness is one of the main challenges in transition from paper- pencil towards computerized adaptive testing (CAT). Aim: This study was aimed at investigating differential item function (DIF), assessing intervening factors in clarifying DIF and suggesting the optimal method for DIF ...
Read More
Background: Test fairness is one of the main challenges in transition from paper- pencil towards computerized adaptive testing (CAT). Aim: This study was aimed at investigating differential item function (DIF), assessing intervening factors in clarifying DIF and suggesting the optimal method for DIF in computerized adaptive testing. Method: The empirical method was applied based on the nature of the study area. Data gathering procedure and manipulating the variables were done using simulation method. The responses of 1000 examinees (reference and focal group with equal 500 numbers) to item bank of 55 dichotomous items were simulated based on 3-parameter logistic model with 20 iterations. Fifteen items were manipulated in terms of DIF type and magnitude and test impact was evaluated based on mean difference of comparison groups. Computerized adaptive test with 30 items was administered via Firestar software package. Analysis was done by logistic regression (LR) and item response theory-likelihood ratio test (IRT-LRT) and the methods were compared based on their power and type I error rate. Results: Type I error rate of likelihood ratio test was less than logistic regression. The power of the methods was influenced by type, magnitude of DIF and test impact. Comparing with logistic regression, Item response theory-likelihood ratio test had more power in detecting uniform DIF for the impact and no-impact conditions and it showed more power by increasing the magnitude of DIF. The two methods showed no difference in assessing non-uniform DIF and both of them were poor. Conclusion: Given the power and type I error rate, likelihood ratio test is an optimal approach in detecting uniform DIF. However, assessing non-uniform DIF requires further investigation.
zahra Bagherikhah; mahbobe Arefi; ehsan jamali
Volume 2, Issue 6 , January 2012, , Pages 1-31
Abstract
Background: The present research has been undertaken with a view to specify the student admission status in Iranian higher education system from the point of view of students studying in some of the state owned universities in Tehran university, professors in Sanjesh organization and some of the related ...
Read More
Background: The present research has been undertaken with a view to specify the student admission status in Iranian higher education system from the point of view of students studying in some of the state owned universities in Tehran university, professors in Sanjesh organization and some of the related higher education officials. Object: The attitudes derived from three sample populations were investigated which revolved around the procedures and the current constituents (including the concentrated testing method, comprehensive examination method, the factor of average written score, the knowledge baseline score, the students quota and students gender) as well as other influential factors (such as non-concentrated examination method, the educational background factor, intelligence and educational aptitude). Method: The research was a descriptive survey. The sample was prepared based on Iran's Morgan table and the research subjects were composed of 346 students from three well known universities such as Sharif Industrial University Shahid Beheshtee University and Allameh Tabataba'i University along with their member board of academicians and a number of the higher education officials. Results: The results obtained in this research are as follows. The greatest percentage of students' responses was related to the concentrated examination method, which showed their relative acceptance; also the highest percentage of professor's response and officials which represented a favorable attitude was to questions related to the concentrated examination method, and non-concentrated examination method which received unfavorable attitudes. There appeared to be no major attitudinal differences among students, university professors and officials towards concentrated examination method, comprehensive examination method, written average score, knowledge score, quotas, and admissions of gender. Conclusion: Based on the views of both students and professors, their responses to questions concerning admission quotas, revealed that some of these quotas resulted in divers complications amongst students including unpleasant feelings, annoyance, and incongruity among students etc. Therefore combining the quotas together or applying them in the form of additional quotas in general entrance exams might prove effective in alleviating anxieties and problems.
